.\" Process this file with
.\" groff -man -Tascii foo.1
.\"
.TH MONETDBD 1 "FEBRUARY 2012" MonetDB "MonetDB Applications"
.SH NAME
monetdbd \- the MonetDB Database Server daemon
.SH SYNOPSIS
.B monetdbd
.I command
[
.I command_args
]
.I dbfarm
.SH DESCRIPTION
.I Monetdbd
is the MonetDB Database Server daemon.  The program is mainly meant to
be used as daemon, but it also allows to setup and change the
configuration of a dbfarm.
The use of
.I monetdbd
is either as user-oriented way to configure, start and stop a database
farm, or to be started from a startup script, such as from
.I /etc/init.d/
on Linux systems or
.IR smf (5)
on Solaris systems, as part of a system startup.
.P
.I Monetdbd
is the system formerly known as
.IR merovingian .
It was renamed to
.I monetdbd
since the name
.I merovingian
proved to be confusing to most regular end-users.  Internally,
.I monetdbd
uses the name
.I merovingian
at many places for historical reasons.
.P
A
.I monetdbd
instance manages one local cluster based, which is a directory in the
system, referred to as the
.IR dbfarm .
Nowadays, the dbfarm location always has to be given as argument
to
.IR monetdbd .
.P
Within its local cluster
.I monetdbd
takes care of starting up databases when necessary, and stopping them
either upon request via
.IR monetdb (1)
or when being shut down.  Client database connections are made against
.I monetdbd
initially which redirects or proxies the client to the appropriate
database process, started on the fly when necessary.
.P
When started,
.I monetdbd
runs by default in the background, sending log messages to
.IR merovingian.log ,
until being sent a stop, terminate or interrupt signal, possibly using
the stop command of
.IR monetdbd .
.P
.I monetdbd
uses a neighbor discovery scheme to detect other
.I monetdbd
processes running in the local network.  Databases from those remote
instances are made available to a locally connecting client.  Remote
databases never override local databases, and their availability is
controlled by the remote
.I monetdbd
process.  See also the sharing capabilities of
.IR monetdb (1)
and the
.I REMOTE DATABASES
section below.
.SH COMMANDS
The commands for
.I monetdbd
are
.BR create ", " start ", " stop ", " get ", " set ", " version ", and " help .
The commands facilitate initializing a dbfarm, starting and stopping the
MonetDB Database Server, and retrieving or setting options.
.TP
.B create dbfarm
Initializes a new database farm, such that a MonetDB Database Server can
be started on that location.  All necessary directories are attempted to
be created, and an initial properties file is created in the directory
itself.
.I dbfarm
must be a location addressable in the local filesystem hierarchy.
.TP 
\fBstart\fP [\fB\-n\fP] \fIdbfarm\fP
Starts
.IR monetdbd ,
the MonetDB Database Server, on the given dbfarm.  When the
.B \-n
flag is given,
.I monetdbd
will not fork into the background, but instead remain attached to the
calling environment, until given a stop signal.
.TP
\fBstop\fP \fIdbfarm\fP
Sends a stop signal to the
.I monetdbd
process responsible for the given dbfarm.
.IP
\fBget\fP <\fBall\fP | \fIproperty\fP[,\fIproperty\fP[,..]]> \fIdbfarm\fP
Prints the requested properties, or all known properties, for the given
dbfarm.  For each property, its value is printed.  Some properties
are virtual, and given for information purposes only, they cannot be
modified using the
.B set
command.
.TP
\fBset\fP \fIproperty\fP\fB=\fP\fIvalue\fP \fIdbfarm\fP
Sets property to value for the given database.  For a list of
properties, run
.BR "monetdbd get all" .
Some properties require a restart of the MonetDB Database Server in
order to take effect.  The
.B set
command, will however always write the property, and tell the running
.I monetdbd
to reload the properties file (if running).  For an explanation of the
properties, see the
.I CONFIGURATION
section below.
.SH CONFIGURATION
.I Monetdbd
reads its properties from the
.I .merovingian_properties
file inside the dbfarm.  This file is created by the
.B create
command.  This file is not meant to be edited manually, instead it
should be updated using the
.B set
command.  The following properties can be set:
.TP
.B logfile
This property points to the file where all log messages are written to.
It is relative to the dbfarm directory, but can be absolute to point to
e.g. another medium.  Changing this property takes effect immediately at
runtime.
.TP
.B pidfile
.I Monetdbd
stores the process ID of the background server in the file pointed to by
this property.  The same rules apply as for the
.B logfile
property.
.TP
.B sockdir
For faster access,
.I monetdbd
uses UNIX domain sockets for its control mechanism and regular database
connections.  The sockets are placed as files in the filesystem
hierarchy.  The
.B sockdir
property controls in which directory they are placed.  In general this
setting should not be changed.
.TP
.B port
This property specifies which TCP port
.I monetdbd
should listen to for connection requests.  Defaults to 50000.
.TP
.B control
For remote management of
.IR monetdbd ,
the
.B control
property specifies whether or not to enable remote management.  Note
that for remote management, a passphrase is required, see below.  It
defaults to false for security reasons.  Changing this property takes
effect immediately at runtime.
.TP
.B passphrase
To control
.I monetdbd
from a remote machine, a passphrase is necessary, to be given to
.IR monetdb (1).
The passphrase can be either given as hashed value prefixed by the hash
type in curly braces (e.g. {SHA512}xxx...) or as plain text value which
will be hashed automatically.  Note that the only hash accepted is the
one specified at configure time, which is @MONETDB5_PASSWDHASH@.
Changing this property takes effect immediately at runtime.
.TP
.B discovery
Specifies whether neighbor discovery is to be enabled using UDP
broadcasts or not.  The broadcasts are done on the same portnumber as
the port setting.
.TP
.B discoveryttl
.I Monetdbd
publishes locally available databases to others periodically.  The
interval used here, defined in seconds, depends on the time-to-live of
the databases before they need to get refreshed.  The default is 600
seconds (10 minutes), which should keep traffic in your network fairly
low.  Additions and removals are processed immediately regardless of
this timeout.  If you are in a network environment where physical
network links disappear often, you may want to decrease this value to
more quickly remove no longer reachable databases.
.TP
.B exittimeout
mservers that were started by the MonetDB Database Server are shut down
when
.I monetdbd
is shut down.  Setting the
.B exittimeout
property to a positive non-zero value will shut down each running
mserver with the given time-out in seconds.  If the time-out expires,
the mserver process is killed using the SIGKILL signal.  A time-out
value of 0 means no mservers will be shut down, and hence they will
continue to run after
.I monetdbd
has shut down.  Note that this particular configuration is extremely
inconvenient.  The default time-out is 60 seconds.  If your databases
are rather large and find your databases consistently being killed by
.I monetdbd
upon shutdown, you may want to increase this time-out.  Changing this
property takes effect immediately at runtime.
.TP
.B forward
.I monetdbd
has two ways in which it can "attach" a connecting client to the target
database.  The first method,
.BR redirect ,
uses a redirect sent to the client with the responsible mserver process.
The second method,
.BR proxy,
proxies the client to the mserver over
.IR monetdbd .
While
.B redirect
is more efficient, it requires the connecting client to be able to
connect to the mserver.  In many settings this may be undesirable or
even impossible, since a wide range of open ports and routing are
necessary for this.  In such case the
.B proxy
technique of
.I monetdbd
is a good solution, which also allows a
.I monetdbd
instance on the border of a network to serve requests to nodes in the
local (unreachable) network.  Note that for local databases, the
.B proxy
method uses a UNIX domain socket feature to pass file-descriptors to the
local mserver.  This effectively is as efficient as the
.I redirect
approach, but still hides away the mservers properly behind
.IR monetdbd .
Hence, in practice it is only relevant for connections to remote
databases to use redirects instead of proxies.  Changing this property
takes effect immediately at runtime.
.SH REMOTE DATABASES
The neighbor discovery capabilities of
.I monetdbd
allow a user to contact a remote database transparently, as if it were a
local database.  By default, all local databases are announced in the
network, such that neighbors can pick them up to make them available
for their local users.  This feature can be disabled globally, or on
database level.  For the latter, the
.IR monetdb (1)
utility can be used to change the share property of a database.
.P
While neighbor discovery in itself is sufficient to locate a database
in a cluster, it is limited in expressiveness.  For instance, database
names are assumed to be unique throughout the entire system.  This means
local databases overshadow remote ones, and duplicate remote entries
cannot be distinguished.  To compensate for this,
.I monetdbd
allows to adds a
.B tag
to each database that is being shared.  This tag is sent in addition to
the database name, and only understood by other
.IR monetdbd s.
.P
Tags are arbitrary ASCII-strings matching the pattern [A\-Za\-z0\-9./]+.
There are no assumed semantics in the tag, which allows for multiple
approaches when using the tag.  The tag is always used in combination
with the database name.  For this, the `/' character is used as
separator, which hence suggests the user to use that character as
separator for multilevel tags.
.I Monetdbd
allows common path globbing using `*' on tags, which allows for many
use-cases.  Consider for instance the following three databases with their
tag:
.PP
.RS 0
dbX/master/tableQ
.RS 0
dbY/slave/tableQ
.RS 0
dbZ/slave/tableQ
.PP
A default match has implicit `/*' added to the search, making more generic
search strings match more specific ones.  Hence, a connect with
database
.I dbX
is the same as
.I dbX/*
and hence matches
.IR dbX/master/tableQ .
Similar, a database connect for
.I */master
matches the same database as before.  Note that the implicit `/*' is
not added if that would cause no matches, such as for
.I */master/tableQ
which would return all masters for 
.IR tableQ ,
which in our hypothetical example is only
.IR dbX .
In contrast, a database connect for
.I */slave/tableQ
matches with either
.IR dbY " or " dbZ .
.I Monetdbd
returns the two options to the client in a round-robin fashion, such
that subsequent connects for the same pattern result in a load-balanced
connect to either of both databases.
.P
With tags in use, one can possibly make distinction between databases,
if setup like that.  The previous example could hence also be setup like
this:
.PP
.RS 0
tableQ/master
.RS 0
tableQ/slave
.RS 0
tableQ/slave
.PP
Connecting to
.I tableQ/slave
would now return either of both databases even though they are not
unique (apart from the host they are located on, which is not shown in
the example).  While being confusing for humans, for
.I monetdbd
it is the same situation as in the previous example.  However, because
globbing allows to make things easier to understand, tags for both
slaves could be changed to
.I slaveX
or
.I slave/X
and use the necessary pattern to match them.  It is up to the user to
decide how to use the tags.
.SH MULTIPLEX-FUNNELS
.I Monetdbd
implements multiplex-funnel capabilities.  As the name suggests two
techniques are combined, the multiplexer and the funnel.
.P
The
.I funnel
capability limits the access to the database to one client at a time.
That is, if multiple clients connect to the funnel, their queries will
be serialized such that they are executed one after the other.  An
effect of this approach is that clients no longer have an exclusive
channel to the database, which means that individual queries from one
client may have been interleaved with queries from others.  This most
notably makes SQL transaction blocks unreliable with a funnel.  The
funnel, hence, is meant to scale down a large amount of clients that
perform short-running (read-only) queries, as typically seen in
web-based query loads.
.P
When a funnel is defined to use multiple databases, the funnel adds a
.I multiplexer
to its query channel.  A multiplex-funnel sends each query to all of
the defined databases.  This behavior can be quite confusing at first,
but proves to be useful in typical sharding configurations, where in
particular simple selection queries have to be performed on each of the
shards.  The multiplexer combines the answers from all defined databases
in one single answer that it sends back to the client.  However, this
combining is without any smart logic, that is, the multiplexer does not
evaluate the query it is running, but just combines all answers it
receives from the databases.  This results in e.g. as many return tuples
for a
.B SELECT COUNT(*)
query, as there are databases defined.
.P
Due to the two above mentioned characteristics, a multiplex-funnel has
some limitations.  As mentioned before, transactions over multiple
queries are likely not to result in the desired behavior.  This is due
to each query to the funnel is required to be self-contained.  Further,
since for each query, the results from multiple servers have to be
combined into one, that query must only return a single response, i.e.
multi-statement queries are most likely causing the funnel to respond
with an error, or return garbled results.  Last, the size of each query
is limited to currently about 80K.  While this size should be sufficient
for most queries, it is likely not enough for e.g. COPY INTO statements.
Apart from the data transfer implications, such statements should not be
used with the funnel, as the results will be undefined due to the
limited query buffer.  Applications using the funnel should aim for
short and single-statement queries that require no transactions.
.P
See the
.B create
command in the
.IR monetdb (1)
man-page for details on how to setup a multiplex-funnel.
.SH SIGNALS
.I Monetdbd
acts upon a number of signals as is common for a daemon.
.TP
.B SIGINT, SIGTERM, SIGQUIT
Any of these signals make
.I monetdbd
enter the shutdown sequence.  This sequence involves cleanly shutting
down listener sockets, shutting down all started databases and finally
terminating itself.
.TP
.B SIGHUP
When this signal is received by
.I monetdbd
it will reopen the logfile as pointed to by the
.B logfile
setting.  Before it reopens the logfile, it will re-read the properties
file from the dbfarm, which might result in opening a different file to
continue logging.
.SH "RETURN VALUE"
.I Monetdbd
returns exit code
.B 0
if it was able to successfully perform the requested action, e.g. start,
stop, etc.  When an error occurs during the action, that prevents
.I monetdbd
from successfully performing the action, the exit code
.B 1
is returned.
.SH "SEE ALSO"
.IR monetdb (1),
.IR mserver5 (1)
